{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting Started\n",
    "\n",
    "### Importing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Computational Graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Const_2:0\", shape=(), dtype=float32) Tensor(\"Const_3:0\", shape=(), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "node1 = tf.constant(3.0, tf.float32)\n",
    "node2 = tf.constant(4.0)\n",
    "\n",
    "print(node1,node2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3.0, 4.0]\n"
     ]
    }
   ],
   "source": [
    "sess = tf.Session()\n",
    "print(sess.run([node1, node2]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, we can add the third node which add node1 and node2.\n",
    "<img src=\"https://www.tensorflow.org/images/getting_started_add.png\" />\n",
    "As it stands, this graph is not especially interesting because it always produces a constant result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Node3: Tensor(\"Add_1:0\", shape=(), dtype=float32)\n",
      "Session.run(node3) 7.0\n"
     ]
    }
   ],
   "source": [
    "node3 = tf.add(node1,node2)\n",
    "print(\"Node3:\",node3)\n",
    "print(\"Session.run(node3)\", sess.run(node3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Placeholder: Used for parameterized to accept externam inputs. A placeholder is a promise to provide a value later\n",
    "<img src=\"https://www.tensorflow.org/images/getting_started_adder.png\" />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "a = tf.placeholder(tf.float32)\n",
    "b = tf.placeholder(tf.float32)\n",
    "adder_node = a + b # provides a shortcut to tf.add(a,b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7.5\n",
      "[ 3.  7.]\n"
     ]
    }
   ],
   "source": [
    "print(sess.run(adder_node,{a: 3, b: 4.5}))\n",
    "print(sess.run(adder_node, {a: [1,3], b: [2, 4]}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also make complex computation graph by using multiplication operation. The preceding computational graph would look as follows in TensorBoard:\n",
    "\n",
    "<img src=\"https://www.tensorflow.org/images/getting_started_triple.png\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22.5\n"
     ]
    }
   ],
   "source": [
    "add_and_triple = adder_node * 3\n",
    "print(sess.run(add_and_triple,{a: 3, b: 4.5}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make the model trainable, we need to be able to modify the graph to get new outputs with the same input. Variables allow us to add trainable parameters to a graph. They are constructed with a type and initial value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "W = tf.Variable([.3], tf.float32)\n",
    "b = tf.Variable([-.3], tf.float32)\n",
    "x = tf.placeholder(tf.float32)\n",
    "linear_model = W * x + b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "init = tf.global_variables_initializer()\n",
    "sess.run(init)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.          0.30000001  0.60000002  0.90000004  1.20000005]\n"
     ]
    }
   ],
   "source": [
    "print(sess.run(linear_model, {x: [1,2,3,4,5]}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loss Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "y = tf.placeholder(tf.float32)\n",
    "squared_deltas = tf.square(linear_model - y)\n",
    "loss = tf.reduce_sum(squared_deltas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see EPE is very high. We can manually adjust the value of W and b to reduce the EPE. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([-0.99999952], dtype=float32), array([ 0.99999833], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "optimizer = tf.train.GradientDescentOptimizer(0.01)\n",
    "train = optimizer.minimize(loss)\n",
    "\n",
    "sess.run(init)\n",
    "for i in range(1000):\n",
    "    sess.run(train, { x:[1,2,3,4,5], y:[0,-1,-2,-3,-4] })\n",
    "\n",
    "print(sess.run([W,b]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "W: [-0.99999791] b: [ 0.99999392] Loss: 2.52847e-11\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "\n",
    "#Model Parameters \n",
    "W = tf.Variable([0.3], tf.float32)\n",
    "b = tf.Variable([0.3], tf.float32)\n",
    "\n",
    "#Model input and outputs \n",
    "x = tf.placeholder(tf.float32)\n",
    "y = tf.placeholder(tf.float32)\n",
    "    \n",
    "#Model\n",
    "linear_model = W * x + b\n",
    "\n",
    "#Loss\n",
    "loss = tf.reduce_sum(tf.square(linear_model - y))\n",
    "\n",
    "#optimizer\n",
    "optimizer = tf.train.GradientDescentOptimizer(0.01)\n",
    "train = optimizer.minimize(loss)\n",
    "\n",
    "#train data\n",
    "x_train = [1,3,2,4]\n",
    "y_train = [0,-2,-1,-3]\n",
    "\n",
    "#initialization\n",
    "init = tf.global_variables_initializer()\n",
    "sess = tf.Session()\n",
    "sess.run(init)\n",
    "\n",
    "# training loop\n",
    "for i in range(1000):\n",
    "    sess.run(train, {x: x_train, y: y_train})\n",
    "    \n",
    "#evaluate training accuracy\n",
    "curr_W, curr_b, curr_loss =  sess.run([W,b, loss], {x: x_train, y: y_train})\n",
    "print(\"W: %s b: %s Loss: %s\" %(curr_W, curr_b, curr_loss))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tf.contrib.learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpxddwzm2j\n",
      "INFO:tensorflow:Using default config.\n",
      "INFO:tensorflow:Using config: {'_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fd69c8bc6d8>, 'tf_config': gpu_options {\n",
      "  per_process_gpu_memory_fraction: 1\n",
      "}\n",
      ", 'tf_random_seed': None, 'keep_checkpoint_max': 5, 'save_summary_steps': 100, 'save_checkpoints_steps': None, '_is_chief': True, '_evaluation_master': '', '_master': '', 'save_checkpoints_secs': 600, '_num_ps_replicas': 0, '_task_type': None, 'keep_checkpoint_every_n_hours': 10000, '_task_id': 0, '_environment': 'local'}\n"
     ]
    },
    {
     "ename": "NameError",
     "evalue": "name 'input_fn' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-45-0171988f7d0a>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     24\u001b[0m \u001b[0;31m# We can invoke 1000 training steps by invoking the `fit` method and passing the\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     25\u001b[0m \u001b[0;31m# training data set.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 26\u001b[0;31m \u001b[0mestimator\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput_fn\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minput_fn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msteps\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1000\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     28\u001b[0m \u001b[0;31m# Here we evaluate how well our model did. In a real example, we would want\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'input_fn' is not defined"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "# NumPy is often used to load, manipulate and preprocess data.\n",
    "import numpy as np\n",
    "\n",
    "# Declare list of features. We only have one real-valued feature. There are many\n",
    "# other types of columns that are more complicated and useful.\n",
    "features = [tf.contrib.layers.real_valued_column(\"x\", dimension=1)]\n",
    "\n",
    "# An estimator is the front end to invoke training (fitting) and evaluation\n",
    "# (inference). There are many predefined types like linear regression,\n",
    "# logistic regression, linear classification, logistic classification, and\n",
    "# many neural network classifiers and regressors. The following code\n",
    "# provides an estimator that does linear regression.\n",
    "estimator = tf.contrib.learn.LinearRegressor(feature_columns=features)\n",
    "\n",
    "# TensorFlow provides many helper methods to read and set up data sets.\n",
    "# Here we use `numpy_input_fn`. We have to tell the function how many batches\n",
    "# of data (num_epochs) we want and how big each batch should be.\n",
    "x = np.array([1., 2., 3., 4.])\n",
    "y = np.array([0., -1., -2., -3.])\n",
    "#input_fn = tf.contrib.learn.io.numpy_input_fn({\"x\":x}, y, batch_size=4,num_epochs=1000)\n",
    "#input_fn = tf.contrib.learn.io.numpy_input_fn({\"x\": x}, y, 4, num_epochs=1000)\n",
    "\n",
    "# We can invoke 1000 training steps by invoking the `fit` method and passing the\n",
    "# training data set.\n",
    "estimator.fit(input_fn=input_fn, steps=1000)\n",
    "\n",
    "# Here we evaluate how well our model did. In a real example, we would want\n",
    "# to use a separate validation and testing data set to avoid overfitting.\n",
    "print(estimator.evaluate(input_fn=input_fn))"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
